Methods and apparatus for creation, distribution and presentation of polymorphic media

ABSTRACT

An apparatus is described for polymorphic presentation ot pre-recorded motion picture content. The recorded content (DVD DISC) comprises individual segments ((1,0) (2,0) etc,}, and the apparatus comprises sequencing means (SEQ GEN etc) for defining a presentation sequence (Xp) of a subset of the segments and for causing the segments to be presented in accordance with the defined presentation sequence. The sequencing means is operable to generate said sequence by selecting and adding segments to a sequence already partly defined, using (i) randomised values and (ii) predetermined rules specifying compatibility between the segments already included in the sequence and segments which are candidates for adding to the sequence. Editing and content capture apparatuses are disclosed, in which a matrix presentation is used to organise content segments. Recorded segments can be grouped in X, Y etc. dimensions and sequences made by varying ‘character’ parameters (Y etc.) as well as timeline position (X).

The invention relates to the creation and presentation of recordedinformation, particularly motion pictures, in polymorphic form, meaningthat the content and sequence of presentation varies from occasion tooccasion.

The inventors are particularly interested in production of motionpictures (‘movies’) which traditionally follow a linear format Whetherissued on polyester film reels or on a digital medium such as thepopular DVD, the selection of scenes and the sequence in which they arepresented is determined initially at the scripting stage but finally andirrevocably in the editing stage, before printing or recording thecontent on the medium on which it is delivered.

The inventors have seen potential in a new form of recording,particularly for audiovisual programmes, in which the order ofpresentation and/or the selection of scenes to include varies, eitherradically or subtly, from viewing to viewing. Precursors of this ideaare known in modern cinema, for example in such films as Timecode andMemento. In Memento we saw the brilliant exploitation of ‘brokentimeline’ techniques. With each scene we are asked to keep a mentalrecord of where we are with the story and let our brains do thereassembling of the scenes into a linear story, meanwhile we are keptguessing for the outcome. Only one timeline is presented, although theproducers have subsequently provided on DVD a linear timeline versionfor comparison with the original.

Timecode aptly demonstrates how the same story can be perceiveddifferently and yet be told at the same time by the clever use of 4simultaneous cameras. It is reported that a music score sheet was usedfor the script.

In practice, however, these “polymorphic” presentations are no more thanalternative cuts determined, as before, in the editing suite at the timethe recording is created. The number of permutations is minimal, andeach is completely determined in advance by the individual author.

Paul Hudak has provided “An Algebraic Theory of Polymorphic TemporalMedia”, Research Report RR-1259, Department of Computer Science, YaleUniversity, New Haven, Conn. 06520-8285, USA (downloaded from Internet).That paper is purely theoretical, however, and no practicalimplementations applications are discussed.

The inventors have identified High Definition digital cinematography andthe Digital Versatile Disc (DVD) platform as suitable technologies uponwhich to base a realisation of truly polymorphic media. It is assumed inthe following description that the reader is familiar with digital movieediting systems, and the DVD system. The DVD system is described forexample in An Introduction to DVD formats by Graham Sharpless (availablefrom Deluxe Global Media Services Ltd. athttp://www.disctronics.co.uk/downloads) and in The Unofficial DVDSpecification, available on the internet from www.dvd-replica.com.Official specifications are available to DVD licensees. Other digitalaudiovisual formats, including downloads to portable media players suchas mobile phones, MP3/MP4 players, are becoming widespread, as well asHD-DVD/Blu-Ray™ formats. DVD will be mentioned below only as arepresentative example of these possibilities.

Patent application US20030194211 of Abecassis describes the variouspossible techniques in the creation and playback of digital recordingsand broadcasts, for example to show more or less detail of some topic inan educational training video recording, according to user choice, or toshow a programme with different levels of swearing and violence and evena different mix of character development versus action, according touser-selected parameters. It is also known that DVDs and digital TVbroadcasts can include alternative camera angles and soundtracks,selectable by the user at the outset or as the action proceeds. None ofthese steps has the intention or effect of disturbing what is anessentially fixed linear sequence of scenes. All the steps arecontrolled by the user, and selected from among a very restricted set ofpossibilities.

Rich file formats exist to preserve edit histories and other ‘metadata’along with the essential A/V content. Some of these are proprietary todifferent manufacturers of editing application software and hardware.There is also the open source AAF (advanced audio file) format, promotedby the AAF Association (www.aafassociation.org). Although AAF files aredesigned to store such metadata in a formed which can be preserved,modified and shared between disparate applications throughoutpost-production, the assumption is still that the file will be‘flattened’ into a linear format before distribution.

In a first aspect, the invention provides an apparatus for polymorphicpresentation of pre-recorded audiovisual content, particularly motionpicture content, the recorded content comprising a plurality ofindividual segments, the apparatus comprising sequencing means fordefining a presentation sequence of a subset of the segments and forcausing the segments to be presented in accordance with the definedpresentation sequence, wherein:

-   -   said sequencing means uses randomisation to define at least part        of the presentation sequence,    -   each segment involved in the randomised part of the sequence has        associated with it at least one rule of compatibility with other        segments, and    -   the sequencing means is operable to generate said randomised        part of the sequence by selecting and adding segments to a        sequence already partly defined, using (i) randomised values        and (ii) predetermined rules specifying compatibility between        the segments already included in the sequence and segments which        are candidates for adding to the sequence.

The sequencing means may be operable to select only a subset of therecorded segments to include in the presentation sequence.

The sequencing means may be operable to vary the relative order ofsegments in the presentation sequence in accordance with randomisedvalue

The sequencing means may be responsive to an OUT rule for a segment, theOUT rule restricting which other segments can immediately follow thesegment.

The sequencing means may be responsive to an IN rule for a segment, theIN rule restricting which other segments the segment can immediatelyprecede.

The sequencing means may be responsive to rules defining a group ofsegments nominally occupying the same position on a first axis, thesequencing means selecting the group according to a first randomisedvalue, and then selecting a particular member of the group according toa second value. These rules allow the author to organise scenes in a‘two-dimensional’ structure. The first randomised value selects thesequence of scenes in a first, ‘X’ axis, corresponding to presentationtimeline, while the second value allows variations of a particular sceneto be selected from different positions on a second, ‘Y’ axis,orthogonal to the timeline. Third and further axes can be providedaccording to the wishes of the system designer.

The second value may be a randomised value or a user-determinedparameter. The different versions of the scene on the Y axis may differin character (level of drama, violence or comedy, for example).

The sequencing means may be responsive to a rule for a segmentrestricting which other segments can appear at any point after thesegment. For example if a character dies in a particular scene, the rulecould exclude all candidate segments in which the same character appearsalive.

The sequencing means may be responsive to a rule for a segmentrestricting which other segments can appear in the same presentationsequence.

The sequencing means may be operable to build presentation sequences ina forward order and/or in reverse order. The building order may beselected based on information stored with the pre-recorded content, forexample on DVD. Other strategies for building the presentation sequencecan equally be envisaged, for example selecting certain key segmentsfirst and then working forwards and backwards from each key segment tocomplete the presentation sequence.

The sequencing means may be operable to apply fuzzy rules, the outcomeof such rules depending on a comparison between the relative strengthsof contradictory rules, or on a comparison of a rule strength and arandomised values. For example, IN rules or OUT rules as mentioned abovemay be defined to provide “weak” or “strong” bonding between twosegments.

The sequencing means may be responsive to further attributes stored inrelation to certain segments.

A segment may have an attribute of dominance, permitting a rule to bebroken according to a strength of the rule.

The sequencing means may be operable to re-use a previously-generatedrandomised value in subsequent decisions between candidate segments. Forexample, the author may wish to structure the movie according to a valuegenerated randomly at the time of playback, but have it influence sceneselection in a consistent way at different points in the presentationsequence.

The sequencing means may be operable such that one or more of theincluded segments is itself defined by a presentation of sub-segmentsassembled at the time of playback. In other words, rather than justassembling complete scenes together in a polymorphic way, the sequencingmeans may be configured to select and manipulate individual shots withinscenes, so that the scenes themselves vary between viewings. Thisincreases the range of polymorphism permitted, while maintaining amanageable structure. All references to ‘segment’ above and below shouldbe interpreted as including references to such sub-segments, unless thecontext requires otherwise.

The sequencing means may be responsive to a rule associated with asegment (or sub-segment) permitting transition to or from the segment ata number of alternative points, selecting the point of transition in thepresentation sequence in accordance with a randomised value.

Such a rule may define a ‘sliding cut’ between two segments, such that asegment is terminated early or late depending on a randomised value, thesliding cut thus having an effect on the total length of thepresentation sequence.

Such a rule may alternatively define a ‘variable cut’ between twosegments, in which the point of transition is varied in accordance withthe randomised value, without altering the overall duration. Thistransition may for example occur between two shots in the videopresentation, while a common audio track continues under both shots.

The sequencing means may be operable to retrieve said rules from specialmetadata fields within an Advanced Authoring Format (AAF) file whichalso defines the corresponding audiovisual content. AAF is a rich fileformat defined to carry metadata detailing the history of a segment ofaudiovisual content, in order that editing and processing decisions canbe retained alongside the finished product. Adopting the AAF file formatallows compatibility with mainstream content creation and productionsystems, while also providing a platform or the addition of metadataspecific to a polymorphic presentation environment.

The invention further provides a recording medium wherein audiovisualcontent and a rules database are stored for use in an apparatusaccording to the first aspect of the invention as set forth above. Thecontent and rules may be stored in one or more AAF files, the rulesdatabase being stored as metadata within the AAF format. The audiovisualcontent may be stored within the AAF file itself, or in separate filesreferenced by the AAF file(s). Conversely, the rules database could bestored in external files, which may be more convenient if thepresentation sequence is being assembled in a broadcasting environment.

The invention in other aspects encompasses scripting and authoring toolsuseful in the production of such recordings.

The invention in a second aspect provides an apparatus for polymorphicpresentation of pre-recorded audiovisual content, particularly motionpicture content, the recorded content comprising a plurality ofindividual segments stored on a recording medium, the apparatuscomprising sequencing means for defining a presentation sequence of atleast a subset of the segments and for causing the segments to bepresented in accordance with the defined presentation sequence, wherebysegments of recorded content can be used and re-used in numerousdifferent presentation sequences substantially without replicating thecontent on the recording medium. This avoids the need to duplicaterecorded content, while storing effectively several different versionsof a movie on the single medium.

The sequences may be pre-defined and stored on the storage medium,and/or they may be generated by a randomisation process in accordancewith the first aspect of the invention.

The storage medium may be a pre-recorded medium like a DVD, or arewritable storage device such as a hard disk drive or solid statememory drive, either built into the apparatus or removable.

Alternatively or in addition, the sequences may be defined by sequencedefinitions received over a telecommunication channel. This optionpermits producers effectively to multiply the content they areproviding, without the need to deliver the bulky data of the contentsegments either on storage device or by download. Users can be givenaccess to new versions of material.

The invention in a third aspect provides an apparatus for polymorphicpresentation of pre-recorded audiovisual content, particularly motionpicture content, the recorded content comprising a plurality ofindividual segments received over a telecommunication channel storedlocally in the apparatus, the apparatus comprising sequencing means fordefining a presentation sequence of at least a subset of the segmentsand for causing the segments to be presented in accordance with thedefined presentation sequence, whereby segments of content storedlocally can be used and re-used in numerous different presentationsequences substantially without re-downloading them via said channel.This reduces channel bandwidth and cost, as well as occupation of localstorage, while permitting the user to purchase several differentversions of a movie on the single medium.

The apparatus may be configured to download a prepared definition of thepresentation sequence.

Alternatively or in addition, the apparatus may include means fordownloading a database of rules associated with the downloaded contentsegments, and for generating new sequence definitions by a randomisationprocess, in accordance with the first aspect of the invention. Theapparatus in that case may be configured to download new rules forpresentation of the same content segments, without re-downloading thecontent segments themselves.

In either the second or third aspects of the invention, the apparatusmay be operable to download additional segments of content from a remoteserver, where a new sequence definition requires content not alreadystored locally in the apparatus, without downloading all the referencedsegments. This may be implemented by the server interrogating theapparatus to determine which segments are already stored when supplyingthe new sequence definition, or by the apparatus itself comparingsegment references in the new sequence definition with the segmentsstored locally.

The invention in a fourth aspect provides an editing apparatus fororganising multimedia content, particularly video data segments, theapparatus have having a user interface providing a matrix structure ofat least two dimensions, in which one or more segments can be assignedto a given location in the matrix.

Segments may be assigned to a location in the matrix by a‘drag-and-drop’ action using a pointing device and a display itemrepresentative of the content in a source location.

Each segment (e.g. a ‘scene’) may comprise an edit of pluralsub-segments (e.g. different ‘shots’, sound elements etc.). Theapparatus may be operable directly to open and make editing changeswithin the segment at a given matrix location, then close and hide theinternal structure of the segment.

The matrix structure may have more than two dimensions, the userinterface presenting selected two-dimensional views according to a pairof dimensions selected by the user. The user interface may providecontrols for selecting the points in a third dimension be represented insaid two-dimensional view.

The apparatus may be operable such that one of said dimensionsrepresents a presentation time sequence. Alternatively, or in addition,the apparatus may be operable to display selected segments in apresentation sequence separate from the matrix, and to permit segmentsfrom the array to be selected and placed at a desired location in thepresentation sequence.

The editing apparatus may include means for automatically generatingpresentation sequence definitions using a succession of co-ordinate setsto reference, in a presentation order, selected segments by theirlocations in the matrix. The sequence generating means may be operableto restrict the permutations of segments included in a sequencedefinition, by reference to compatibility rules associated with one ormore matrix locations. The sequence generating means may be operable toinfluence the selection of segments to be included in the sequencedefinition, by reference to preference values indicated by the operator.The sequence generator may be responsive to preference expressed interms of one of said matrix dimensions. The sequence generator mayinclude a randomiser for use in determining a selection, taking accountof any such restrictions and/or preference values expressed.

The apparatus may include means for reproducing the selected segments inthe presentation order as a continuous presentation, for immediateviewing or in a recorded format.

The apparatus may include means for storing a plurality of alternativesequence definitions on a storage medium, together with content for atleast the set of segments necessary to reproduce at a later time thesegments referenced in a selected one of said sequence definitions.

The matrix structure may be defined by metadata in one or more AAFformat files. Each segment may comprise an AAF file in itself. Data forplural segments may be contained within one AAF file.

The invention in a fifth aspect provides an apparatus for organisingmultimedia content during creation, particularly video data segments,the apparatus have having a user interface providing a matrix structureof at least two dimensions and having a communication interface to videorecording apparatus, whereby a segment can be assigned to a givenlocation in the matrix immediately at the time of recording.

The apparatus may be operable such that an operator selects a matrixlocation using a pointing device prior to initiating capture of asegment of video recording. The apparatus may permit a plurality oftakes to be associated with a given matrix location. This may be adedicated feature at each matrix location, or one dimension of thematrix might be used to identify plural takes. The apparatus may beoperable to present the takes for selection of a preferred take at eachmatrix location. The apparatus may be operable to retain references tothe other takes, for future access.

The user interface may be presented so as to highlight automaticallymatrix locations for which content is still to be captured.

The matrix structure may have more than two dimensions, the userinterface presenting selected two-dimensional views according to a pairof dimensions selected by the user. The user interface may providecontrols for selecting the points in a third dimension be represented insaid two-dimensional view.

Each segment (e.g. a ‘scene’) may comprise an edit of pluralsub-segments (e.g. different ‘shots’, sound elements etc.). Theapparatus may be operable directly to open and make editing changeswithin the segment at a given matrix location, then close and hide theinternal structure of the segment.

The apparatus may be operable to store compatibility rules betweensegments associated with specific matrix locations prior to capture ofthe content forming those segments, said rules being suitable toindicate permutations permitted to preferred in subsequent polymorphicpresentation of the segments. The apparatus may provide for editing ofsaid rules before and after capture.

The invention provides methods of capturing, editing and distributingaudiovisual content using the apparatus of the various aspects of theinvention as set forth above.

While the above aspects of the invention are expressed in application toaudiovisual content (video and audio content), the same principles areapplicable to textual matter (literature). The invention thereforefurther includes aspects corresponding to each o the aspects set forthabove and described in the examples below, but where the audiovisualcontent is replaced by written matter. Literary authors at many timeshave experimented with fractured chronology, and presentation of a storyfrom the perspective of different protagonists. As with filmproductions, however, the invention permits such experiments to beretained and played out for the user in a seamless presentation, ratherthan the author having to select and enforce one choice from among themany alternative structures. The invention thus enables and encompassespolymorphic ‘e-books’, as well as polymorphic movies.

The invention in its various aspects can be implemented by dedicatedhardware or a combination of software and hardware such as PCworkstations (at the production side) and DVD player subsystems (at theconsumer side). The invention is not limited to any particular one ofthese implementations, except where this is explicit in a particularclaim.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way of exampleonly, by reference to the accompanying drawings, in which:

FIG. 1 illustrates the structure of a segment of audiovisual materialrecorded as part of a Polymorphic Movie in accordance with the presentinvention;

FIG. 2 shows a known Edit Decision List useful in understanding thepresent invention;

FIG. 3 illustrates the principles of two-dimensional permutation ofscenes or segments in a Polymorphic Movie;

FIG. 4 shows an example script and presentation sequence;

FIG. 5 shows a Wildcard segment added to the script of FIG. 4;

FIG. 6 shows schematically the architecture of a system for producingand playing Polymorphic Movies in one embodiment of the presentinvention;

FIG. 7 is a flowchart showing operations performed by the player of FIG.6;

FIG. 8 shows the set of another example polymorphic movie, The NextRoom;

FIGS. 9 and 10 show example user interface screens for a matrix editoruseful in the development of polymorphic movies, based on an example ofa pop music promotional video;

FIGS. 11( a), (b), (c) and (d) illustrate different mechanisms fordelivery of polymorphic movies to different types of user apparatus, andwith different payment models;

FIG. 12 illustrates the structure of files in an AAF-compatibleembodiment of the invention;

FIG. 13 shows schematically the architecture of an AAF-compliant editingapparatus in the preferred embodiment; and

FIG. 14 shows an example user interface screen for the editing apparatusof FIG. 13;

FIGS. 15 and 16 show sliding cut and variable cut functions available ina preferred embodiment of the present invention; and

FIG. 17 illustrates the principle of a semi-automated content capturetool an process, useful particularly but not exclusively in theproduction of polymorphic movies.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Segments have rules associated with them that are stored in a database.These rules may or may not be supplemented by a basic set of rulesprogrammed into the player. The rules database can be stored as oneentity, or it may be distributed so that rules are attached to andhandled together with data defining the content of each segment. All thesegments and rules are preferably stored together on the same recordingmedium such as a DVD, however, in order that the polymorphic movie canbe distributed conveniently. (An example player architecture will bedescribed later, with reference to FIGS. 6 and 7.)

An interesting starting point to consider the structure of the rulesdatabase for polymorphic presentation of these segments is a structureused in standard commercial software for editing digital movies. It'scalled Edit Decision List (EDL).

Components of an EDL

FIG. 2 shows an example of an EDL from the editing product AdobePremiere. While slight differences exist among different EDLs, mostcontain eight primary columns, two auxiliary columns, and the followinginformation, labelled A, B etc. in the Figure, as follows:

-   -   A=Header: Names the list and the type of timecode in which the        record was created (drop-frame or non-drop-frame).    -   B=Event Number: Identifies a single event or edit. Unnumbered        lines accompanying events are called notes or comments.    -   C=Source Reel ID: Identifies the name or number of the videotape        containing the clip.    -   D=Edit Mode: Indicates whether the edits take place on the video        track only (V), the audio track only (A), or a combination of        both (B).    -   E=Transition Type: Describes the type of edit: C represents a        cut, W represents a wipe, K represents a key (superimposed), and        D represents a dissolve.    -   F=Source In and Source Out: Lists the timecode of the first        frame and the last frame of the clip as it appears on the source        videotape.    -   G=Program In and Program Out: Lists the timecode at which the        source clip is to be recorded on the master tape.

Descriptions Extracted from Adobe Premiere™ Help File

In the conventional production process, the edit decision list is takenas instructions for a further module to concatenate the various scenes,with the prescribed transitions, sound track and so forth, to produce afinished presentation on tape, film or DVD. The edit decision list isthus a part of the production process, not a part of the finishedproduct. In film, the product might be divided into smaller chunks so asto fit on a number of standard reels. In DVD, the producer can dividethe presentation into chunks and define a number of entry pointsaccessible via a scene menu. Generally, however, the sequence ofpresentation is very rigidly defined and fixed on the storage medium.Even in the AAF format (described in more detail below), where editinghistory is preserved in a file package to be accessible throughout thepost-production process, the assumption is that such data will strippedout before product is released on DVD or other format. In contrast, thepresent inventors propose to include in the DVD product the “rawmaterials” on the list together with an enhanced form of edit decisionlist (incorporating the rules database). Together with an enhancedplayer (or possibly enhanced programming on the disk itself), thisallows polymorphic presentations whereby the recorded scenes can beselected, ordered and concatenated into a seamless presentation, all atthe time of playback, rather than at the time of production. It is notedthat many existing DVD releases include additional material such asalternative endings, ‘deleted scenes’ and so forth, but these arepresented as separate items to be selected and viewed independently andout of context. This does not give the viewer a real opportunity toexperience the complete presentation, seamlessly in its alternativeform.

Even without each user having an enhanced player, the rules database maybe held by a content provider and/or distributor, who uses it togenerate the presentation sequence and broadcast, stream or download thefinished presentation to an end user apparatus. The end user apparatusin that case might be a simple TV set or computer video player.

Referring to FIG. 3, in the system proposed herein, the creation of anew type of Polymorphic Movie is illustrated by visualising theavailable segments as being arranged on two or more ‘axes’.

In the illustration, the lower horizontal axis labelled XP (thepresentation axis) corresponds effectively to the timeline of the finalpresentation: a segment is presented to the viewer immediately after theone to its left along the XP axis. The segments are for convenienceshown all the same length but will not generally be so. In the desiredpresentation there are eight segments, just for the sake of example,numbered 1-8.

The curved arrows represent some steps in the process implemented by asequencing device or process within the player, selecting the segments1-8 from a pool of available segments (the source material) shown in atwo-dimensional array in the upper part of the drawing. Thistwo-dimensional arrangement is a useful structure for authoringpolymorphic movies and imposing some constraints on the randomisationprocess at a fairly high level and in an intuitive manner, rather thanexplicitly defining every rule individually. Nevertheless, as aconsequence of arranging the source segments in this way, rules areimplicitly generated and stored with the source materials in a form of adatabase, which is the enhanced edit decision list. It is a matter ofdesign choice, whether to store the resulting rules explicitly in amatrix format, or to leave the matrix structure as a convenient view forthe author, not explicitly represented within the rules database itself.A description of the apparatus involved in the creation and playback ofthe polymorphic movie will be described later.

In the “script domain” shown in the upper part of the drawing, there isan axis XS, along which segments (scenes) available for inclusion in thepresentation sequence are arrayed. Orthogonal to the XS axis, the scriptalso arranges scenes (segments) on a Y axis. Each segment thus has anindex X,Y, starting from 1,0 up to n,+2 in the illustration, by which itcan be identified in the rules database, and through which it can befound on the storage medium when required for playback. The X axis canbe viewed, if desired, as corresponding to order in the presentationsequence.

However, ordering of the segments on the XS axis does not directlyconstrain their ordering on the presentation axis XP. To do this, adatabase rules is created during the production processes, whether inpre-production (for example scripting, storyboarding) or post-production(for example editing). Similarly, the position of a segment on the Yaxis is represented in the rules database stored with the segments. Notethat all positions in the X, Y array need not be filled.

As mentioned already, the rules provided for each segment willdetermine, whether its position in the presentation sequence XP isfixed, free subject to certain constraints, or completely free. Withinthese rules, a randomising process performed at playback time willdetermine exactly which segments are reproduced for the viewer, and inwhat sequence. The most basic type of rule describes the “bonding”between segments, which is analogous to the ability of atoms andmolecules to bond with each other to form new molecules. For eachsegment there will typically be a rule listing or otherwise definingwhich other segments are permitted to follow immediately in thepresentation sequence (OUT rules). There may also be IN rules specifyingwhich other segments can precede the present one.

Some examples of the types of rules which can be coded in the databaseare illustrated graphically in the upper part of the drawing as follows:

-   -   Using double-headed arrows it is indicated that segment 2,+2 can        only be followed by segment 3,+2, and segment 3,+2 can only        follow segment 2,+2 in the presentation sequence as shown, the        sequence generator is forced by the rules to place segment 3,+2        at position 5.    -   Similarly, segments 4,0 and 5,0 are bound together, but with a        weaker connection (thinner arrow). This allows other mechanisms,        to break the bond, as described further below.    -   Using a solid dot at the IN side of segment 1,0 it is indicated        as a rule that segment 1,0 must be the first segment in the        presentation sequence.    -   Similarly, solid dots at the OUT side of segments N,−1 to N,+2        indicate rules that each of these segments can only appear as        the final segment in a presentation. The fact that there are        four of these indicates that at least four alternative endings        are possible for the same script, in the polymorphic        presentation.

The drawing also shows how the content can also include variants of agiven segment, visualised on one or more orthogonal axes, which can beselected according to a further randomised value. Segments can begrouped having the same X value, but different Y values ascribed to them(−2, −1, 0, +1, +2, for example). Then, once the group has been selectedfor inclusion in the presentation, a randomised value of Y is generatedand used to select exactly which one of the segments should be includedin the presentation sequence, at that point on the X axis/timeline. Thesame value of Y may be used at different points in the sequence, ratherthan being randomised at each use. In some dimensions, the value maysimply be a user-determined parameter, it may function similarly to the“character” controls in the Abecassis proposal system mentioned above.Where it includes a random element, the variations in the content willproduce results more challenging to the viewer, and the author. As asimple illustration of the potential for creative expression, we canimagine that an author might for example use increasingly positive andnegative values of Y to represent increasingly positive and negativemoods in the content. The feel of the scenes and ultimately the courseof the plot can then be varied under control of this parameter Y.

Although only two axes are represented in the drawing, there is no limitto the number of different dimensions provided in which to organisevariations. Additional axes can be defined which are regarded asorthogonal to the X and Y axes, and as many as desired. It is free forthe writer and director to develop their own scheme of variations, andto label the axes how they feel is right, according to their own styleand the effect they wish to achieve in a particular project. One exampleof another axis might be a soundtrack or dialogue axis. For example a“voiceover” might be included in a scene or not. Because 100% randompresentations are unlikely to be satisfying to either the author or theaudience, these axes represent one way of structuring and visualisingthe constraints on the randomisation process that is at the heart of thePM presentation. In a computer-assisted scripting & productionenvironment, a similar display may be presented to thewriter/director/editor et al, but the invention is not limited to suchimplementations. An illustration of a possible user interface forediting polymorphic movies is given below, with reference to FIGS. 10and 11.

With regard to the length of the presentation sequence, the rules forthe sequence generation process may dictate that a presentation of N=8segments is to be generated, a sequence of a certain duration, or Nitself may be a randomised variable.

Example Script

In order to put the above discussion into context, FIG. 4 presents avery small example of a two-dimensional polymorphic script, involvingcharacters Andrew, Angela, Neil and Sophie. In the example, there arefour scenes along the X axis, and three Y axis variants at each Xposition. The writer has chosen to organise the scenes such that thesetting of the scene (interior, exterior, location etc.) is the same forall the variants for a given XS, but the characters and/or actionchanges according to the Y index.

In the presentation sequence presented along the axis XP, randomselection in the Y dimension has resulted in the inclusion of segmentsin the sequence: 1,1; 2,2; 3,0 and 4,1. The presentation ordercorresponds exactly to the order along the XS axis, either because arule XP=XS has been established by the writer, or just as a result ofchance.

Even with such a rigid rule, and with only 4 segments in the finalsequence, 3×3×3×3=81 different presentations can result simply fromrandomising Y at each step. One can see that the architecture of a PM isreally quite simple, but it already offers multiple, non-lineardirections and outcomes. Provided each cell or segment packet retainsits ‘morphic’ qualities and ‘morphing’ characteristics, and provided itretains its links to allow ‘bonding’, the author and playback systemtogether can guarantee a seamless outcome and faultless play.

The axes offer great scope in terms of scenes and sequences by addingcharacter and ability to morph; some additional attributes can be addedto some or all scenes whereby the author can free and/or constrain therandomisation process at a higher, more structural level. This willassist the author in achieving presentations which, although having arandomisation process at work in their definition, still manage toconvince the viewer with a sense of coherence. The options forcreativity will be far greater in that case than if presentations aretruly random, or constrained only by what scene can come immediatelybefore or after another. For each of these the database includes anindication that the scene has a particular attribute, and one or moreparameters specifying the detail or effect of that. The range of thesespecial rules is still being developed, and imagination will providemore once experience of PM movie production is more widespread. For thetime being the following examples will give a sense of thepossibilities.

Wildcards

FIG. 5 illustrates a further addition to the simple X,Y arrangement ofsegments, involving a ‘Wildcard’ which is a single segment outside thegeneral X-Y structure. In the example, the wildcard segment W1 shows thecharacter Neil dead.

For the sequence generator, segments identified as Wildcard segments arenot fixed on any axis (particularly the time or X axis) but rather have“roaming” characteristics. The rules for other scenes may be such thatthe inclusion of a Wildcard scene has a far-reaching impact on thepossibilities at other parts of the presentation. For example, if theoverall story is about murder and mystery, a touch of comic action mightturn the film into black comedy. On the other hand there might besituations where a Wildcard might completely destroy a sequence. If aWildcard is indicated by the randomiser, the Sequence Generator willdetect this and decide whether to retain the outcome or discard it bygoing through another randomisation or even drop some scenes and callfrom a library for others.

In the case where the Wildcard in which Neil is dead has been includedin the presentation sequence, the reader can readily envisage rulesshould then apply to prevent scenes where Neil is obviously aliveappearing at a later time.

Wildcards are best planned rather than left to chance. They have thesame construction as a normal segment with the same IN/OUT rules forexample. There is no reason why Wildcards should be casual or unruly,unless of course the story or the authors demand this for effect or wejust want to experiment. As a very simple example, most filmstraditionally have a title sequence before the action. However, plentyof films are known where a ‘pre-title’ sequence is included, which canbe short (as in James Bond films) or long, up to the complete length ofthe presentation (Apocalypse Now). In the PM format, the title sequencecan be defined as a Wildcard or series of Wildcards, free to appearafter other scenes. Rules can be defined so that the titles are kept insequence relative to one another but can be separated by different othersegments.

In other words, just like viruses in humans or computers can destroy aperfectly healthy body and vaccines can restore or increase immunity,here too a Wildcard has the power to destroy a ‘romantic story’ bybringing in elements of murder and mystery and turning the story in adrama or thriller and vice versa, depending on the power (‘dominance’)of the cell and its inclination (‘character’).

Segment Bonding

Each Segment has to have the ability to recognise whether it can‘concatenate’ or not. In this case the ‘IN/OUT’ procedures refer to adefinition, script or library determining the extent to which theSegment itself can or cannot be used in any given sequence. The abilityof a Segment to interlock itself into a sequence is called ‘bonding’and, as the word implies, the definition for this characteristic mightinclude ‘strong’ and ‘weak’ variables, which will make the ‘attack’ by aWildcard more or less successful. In FIG. 3 the strengths of bonds wereindicated visually by different weights of arrows. This device could beused also in a screen display of a scripting system (or on cards if thatis how one is organising one's thoughts. It will be appreciated thatbonding is a concept which can be implemented through a variety of ruletypes, ranging from the most basic IN/OUT neighbour rules tolonger-range relationships, fuzzy (strength) relationships, wildcardsand so forth.

Segment Dominance

An attribute of Dominance can be assigned to a Segment in the PMdatabase, to determine its dominance over other Segments that may becandidates suggested by the randomiser. For example in a Drama, theremay be very strong dramatic scenes which will seldom allow change intoanything else in that particular sequence or even for a number ofsequences before or after the dominant scene. This will enable a degreeof control over the structure of any PM since a number of scenesdesigned this way will in themselves deter any Wildcard or Segments frominterfering with the flow of the sequence or the Story.

Segment Character

Segments can be assigned a Character, representing the inclination ofits content in some pre-defined classification. If the story is comic innature and then the character of the scene will reflect that andinfluence the positioning and usage of the Segment. Sometimes, duringfilming a conventional movie there may be ‘cuts’ not necessarily shownfor one reason or another. Often this is because they have assumed acharacter which is inappropriate in the context of the normal movie. Ina normal movie these unused pieces of polyester will end up at best asadd-ons on a DVD. In a polymorphic movie, on the other hand, thecharacter of the presentation may in fact vary, these scenes could henbe judged compatible and therefore be used (allowed to appear). In ourcase such Segments are digital blocks, which can be introduced in astory at any time as and when they are required, nothing is discarded.This will give greater flexibility to the production of PMs.

Other relationships that may be controlled between and within sceneswill be illustrated below in the description of the AAF-compatibleembodiment (FIGS. 9-13).

Physical System Architecture

It is conceivable that some of the novel functions described herein mayin fact be implemented with advanced programming techniques within theexisting DVD command set. However, limitations in the complexity of thatcommand set, the number of commands recognised and the size of programspermitted by the DVD-standard “virtual machine” are likely to make atruly seamless polymorphic presentation difficult to achieve without anenhanced playback apparatus. In any case, even if DVD players aretheoretically capable of some of the effects described herein, currentauthoring tools do not, to our knowledge, facilitate the creation ofDVDs having the appropriate databases and programming for trulypolymorphic presentations. For example, it is typical for “studiorelease” and “director's cut” editions of the same material to beproduced side by side on one or more discs, even though 90% of thecontent is identical. Whether this is due to limitations in the DVDplatform itself, in the authoring tools or in the imagination of theproducers, this duplication is clearly wasteful, and seriously limitsthe number of polymorphic variations that can be produced in areasonable package. The inventors therefore propose that the content bestored in separate segments on the disc (or other storage medium) andconcatenated into a particular presentation sequence only in the playerapparatus. Provided minimum disc access times are respected, and/orsufficient buffering is provided, segments which are suitably crafted tolead into one another can be edited seamlessly into numerous differentsequences, all without detracting from the cinematic experience.

Having said that special playback apparatus is likely to be required, itis envisaged that a conventional DVD storage and player architecture canbe used as the platform for such an enhanced player. In particular, oncethe desired presentation sequence has been defined, it can berepresented in the form of conventional DVD-standard “program chain”(PGC) entities. Each PGC has PRE and POST commands (procedures) whichare executed by the “virtual machine” within standard DVD player. Whilein a normal DVD recording these would be pre-defined by the author andstored with the content files on the DVD medium itself, PGC entities inthe same standard format can be generated with PRE and POST proceduresappropriate to a new randomised sequence generated immediately prior toplayback by the polymorphic movie sequence generator, and then fed tothe normal DVD playback modules to deliver the presentation to the user.It is important in most cases that a “seamless” transition from onescene to another be achieved. The “cinematic” experience generally willbe spoiled for viewers if the presentation pauses while files are loadedbetween randomised segments. Care must be taken in the authoring processto ensure that this is possible within the constraints of the DVDsystem. Alternatively, enhanced hardware with faster processing,increased buffer capacity and so on may be preferred to relax theauthoring constraints and permit the maximum freedom in scripting andpresentation.

FIG. 6 shows schematically the architecture of the authoring process forpolymorphic movies and the enhanced playback apparatus according to oneembodiment of the invention. The component DVD VM (virtual machine) isprovided by the microcontroller with RAM memory and firmware (ROM) thatare commonly provided in a DVD player. Likewise the decoder for ANcontent is the same as in any DVD player. The additional componentsinclude the Sequence Generator responsible for generating a presentationsequence defined not by data taken from the DVD itself, but by arandomisation process which refers to rules stored alongside theaudiovisual content on the DVD, and extracted into a database, alsoshown. A special menu is provided by module PM Control (via userinterface USER I/O) for the user to interact with the sequence generatorand control the polymorphic playback features. Based on the user input,the rules, and random numbers generated on demand by a randomiser RNDwithin the sequence generator, the presentation sequence is defined, anddata to cause the presentation of the stored AV content in accordancewith that sequence is loaded into a buffer for supply to the DVD virtualmachine. A broken line shows how similar data might be transferreddirectly to the same virtual machine for playback of conventional DVDcontent. Other functions, such as the conventional menu, are notillustrated for simplicity.

The additional components described may be provided by additionalhardware physically connected between the disc reading hardware and themicrocontroller. Equally the additional components and the “existing”DVD virtual machine can be implemented physically within the samemicrocontroller, ROM and RAM. Needless to say, the processing andstorage capacity of those components may need to be increased in orderto incorporate the added functions.

FIG. 7 shows the operating sequence when a disc is inserted in theapparatus by a user. To the left of the flowchart appear the names ofthe components of the system primarily in involved. The user with aremote control handset turns the player on, and the disc is read. The PMControl module checks whether the disc carries a conventional orpolymorphic (PM) recording. If the disc is not a PM recording controlpasses to the normal control functions of the DVD player, wherein thevirtual machine (VM) controls the presentation of content from the discin accordance with PGC data retrieved directly from the disc.

If the disc does carry a PM recording the PM Control module displays aPM menu or menus by which the user can select a “normal” viewing of thedisc, or can select a randomised PM presentation. Parameters such as the“character” of the movie desired can be set through these menus, whichnaturally have to be defined as part of the scripting and authoringprocess, if they are to be available to the end user at this stage.Assuming a PM presentation is selected, the sequence generator gets towork in accordance with the user-selected parameters and the rulesdatabase retrieved from the disc, to load a brand new, randomisedpresentation sequence into the buffer memory of the DVD player. Controlis then passed to the DVD virtual machine which controls playback ofstored content from the disc just as if the presentation had beendefined on the disc by a conventional authoring process.

As described for example in The Unofficial DVD Specification, mentionedin the introduction, the DVD virtual machine provides a number ofprogramming instructions (the DVD VM Command Set) that are used tocontrol the sequence of presentation of a movie from files stored on aDVD. These commands are used in particular to present menus and allowusers to select different material to view besides the main feature on adisc. Limited interactivity can be provided, sufficient for simplegames. Functions such as comparison of numeric variables and generationof randomised values are included in the standard DVD VM command set.Given sufficient processing power and memory, those or equivalentcommands could be used in implementing the PM Control and SequenceGenerator functions in the enhanced player. For polymorphicpresentations, additional commands may be added to create a command setwhich is a superset of the DVD VM command set. Table 1 below gives someexamples of additional commands that might be useful. These are only afew examples from what would be a larger command set in practice. Theexamples include high-level procedures and also low-level steps.

TABLE 1 Command Description 1. Context Check story context, that is tocheck whether scenes of a certain character should be favoured in thesequence generation. 2. Bond w Fuse two scenes together (weak) - part ofsequence generation 3. Find Child Find relative scene ‘down’ - used incomplex rules 4. Find Parent Find relative scene ‘up’ 5. Find IN ‘x’Find IN characteristic ‘x’ - find all scenes that can follow scene ‘x’6. Concatenate A high-level procedure to generate a linked sequenceready for playback 7. Permute A high-level procedure to generate apresentation sequence by repeated randomisation and application of therules from the database 8. Backtrack A lower level command allowing asequence generation process to be partially undone, for example becauseunresolvable conflicts between rules. 9. Save A command to save agenerated sequence for playback at a later date (for example on usercommand)

It is proposed in the apparatus above that the Sequence Generator shouldoperate so as to generate the entire presentation sequence prior tocommencing playback. As an alternative, the presentation sequence couldbe generated as the presentation progresses, possibly influenced by userfeedback (imagine a “Boring” button on the remote control which triggerssome change in the parameters, or triggers the inclusion of a disruptiveWildcard scene). Defining the presentation sequence on the fly probablyrequires that the sequence is generated by an algorithm that starts withthe first segment to be presented and proceeds forwards in time. Ingeneral, however, a variety of different algorithms can be envisaged, asmentioned in the introduction.

Authoring Polymorphic Movies—Example: The Next Room

At the head of FIG. 6 one can see the steps of content creation (filmingscenes with sets, actors, lighting etc.) and scripting which feed theninto the step of authoring the polymorphic recording itself, be it onDVD or some other medium. There are undoubtedly challenges ahead indeveloping powerful, flexible and intuitive authoring tools tofacilitate the creative process and automate as much of the technicalprocess as possible. A more detailed authoring system will be describedbelow. For the present example, it is sufficient to know that its basicfunctionality is to produce a set of content segments and associatedrules database which can be interpreted by the player described above.

Experience with a small, experimental polymorphic film entitled The NextRoom gives some pointers to the type of considerations that arise in thecreative process.

FIG. 8 is a photograph of the set consisting of five identical rooms. Byediting scenes together as actors pass between the rooms, the action iseffectively set in an infinite landscape of identical rooms. In The NextRoom a particularly regular geometry and structure for the set and thescenes is chosen so that the scenes can play seamlessly in any sequence.A man and a woman are the principle characters.

Stepping into a film studio and getting down to filming is never easyand requires a lot of planning, visualisation, control, checking,reviewing and so on. Creating a PM is just as demanding. Here theattention to detail becomes essential. Each scene has to be treated as amovie in its own right, from beginning to end. Whilst Actors are stillafforded a degree of freedom in the ‘middle’ or ‘body’ of any scene theall important IN and OUT takes for any segment need extreme care andattention to structure.

Some extracts from the script will give a flavour of the considerationsat work in this new creative environment:

-   -   All rooms are square and of the same construct. Consisting of        four equilateral walls, each with a centrally placed double        swing door. Both the MAN and the WOMAN are breathless and        anxious looking upon entering and leaving each scene. The woman        perhaps a little more panicked. The actors are required to bring        a look of ambiguity that could be taken as both hunter and        hunted.

Special consideration was given to the choice and position of thecameras and medium to be used: Three High Definition Camerasstrategically placed in each room. Wide shots were always used for theOUT/Exit takes.

An extract from the script for four individual scenes (segments) thenreads as follows:

-   -   BLACK SCREEN FADE-IN (only and always on the following scene)    -   INT. NEXT ROOM 1    -   CAMERA P.O.V. Bursts through the swing doors and heads straight        to the doors opposite and bursts through them. No other doors        are swinging and no o.s. noise is heard.    -   INT. NEXT ROOM 2. The doors swing open as the WOMAN bursts into        the room. She stops and looks at each of the doors, anxious and        confused, which one to take, she runs to the swing doors in        front and bursts through them into the next room.    -   INT. NEXT ROOM 3. A MAN bursts through the swing doors and into        the room. He stops and looks at the doors. Anxious and confused        he sees the ones in front are swinging. He runs to and bursts        through them.    -   INT. NEXT ROOM 4. The WOMAN bursts into the Room. She looks        panicked and confused. She looks behind her, at the doors she's        just come through, fearful of something. She takes the left door        and bursts through it

Although these scenes are numbered, the scripting, pre-production andproduction process must take into account that they might be presentedin any order. It has been found that cards (either real physical cardsphysically manipulated or virtual cards manipulated in a computersystem) are a useful device. Each scene can be sketched on its card inthe manner of a conventional storyboard. The cards can then be shuffledto try the effect of random permutations on the story, on continuityetc.. Rules for bonding etc. can be recorded on the cards or in aseparate document.

In the particular case of The Next Room there are 19 principal scenesmaking up the story. The scenes were crafted specifically with 100%polymorphism in mind, meaning that the order of the scenes is completelyrandom. The only rule applicable in that case was that each scene shouldappear once in the presentation sequence. We have calculated that 19separate scissor-cut scenes can be shuffled into a new script give atotal number of 121,645,100,408,832,000 different permutations, all fora film that is approximately 3 to 4 minutes duration. In fact it wouldtake approximately 806 billion years of continuous viewing to see everydifferent permutation. Accordingly, even if the degree of permutationpermitted in a different project were limited to very few scenes, orconstrained by far stricter rules, it does not take much freedom at allto realise a vast number of permutations, to create a movie that willnever be the same in any two performances. The additional dimensions Y,Z etc add to the number of permutations possible. (As an aside, theplayer may store the generated sequences automatically or at userrequest, so that exactly the same presentation can be viewed again orshare with others. This would be a matter for the author and playerdesigner to permit if they want.)

Given the ‘pure’ nature of the story the scenes end up looking verysimilar to the untrained eye. This is purposely so because the authorDas Abra wanted to eliminate any ambiguity or clutter which could resultin us missing the point of the exercise. The rooms are empty; thereforethe story is devoid of any reference to material that would defocus ourattention from the form of each room, representing a scene or cell, andthe mechanisms which link the rooms, the doors, or in our case theIN/OUT procedures. The actors play ambiguously to illustrate the abilityto morph at any time the sense, feel and character of each random story.

Music

While the dialogue and effects elements of the soundtrack of a scenewill generally be stored as part of the content, it is difficult topermit randomisation and yet keep a musical accompaniment flowingsatisfactorily. In order to address this, it is envisaged that musicalscore will be separated from the segments themselves, and played forexample through a MIDI-based synthesiser in the player (similar to acomputer sound card). The character of the music can be changed on thefly in response to the character of variants selected in the Ydimension, for example.

The examples of FIGS. 4, 5 and 8 illustrate the use of polymorphism to“play” with the structure of a presentation for novel dramatic orartistic effects. The reader will appreciate that the development ofscripts for such material which will yield truly satisfying results isnot a trivial business. Other applications such as promotional videosfor pop music may also be considered, however, and show more immediatecommercial potential. In a typical music video, the basic content is aperformance of the song, synchronised with the sound track. The imagesare taken, however, from many different shots, with different locations,angles, focus on different members of the band and so forth. There mayor may not also be one or more “mini dramas” performed which are notnecessarily synchronised to the musical performance, and of course theremay be any number of other images, limited only by the imagination ofthe artists and video author.

Such commercial audio-visual productions lend themselves very readily tothe multi-dimensional polymorphism described above, and especially tothe matrix representation. For example, with the x axis againrepresenting the presentation time line, the x axis representing, forexample, focus on different members of the group, and the z axis forexample, representing different locations where images have been shot.

Matrix View Editor

FIGS. 9 and 10 illustrate a possible graphical user interface for the“super editor” which manages data in the assembly of a polymorphicpresentation based on X, Y and Z axes, explicitly representing theassembled material in that matrix form. Referring first to FIG. 9, themain area of the display is occupied by icons, possibly includingthumbnail images, but in any event representing individual imagesequences (these may be individual shots or pre-assembled scenes). Alongthe foot of the display, audio tracks are represented, divided intosegments each corresponding to one step along the x axis. Note that,whereas a dramatic presentation can in principle have scenes swapped inorder, or scenes of different lengths, the video segments arranged incolumns X1 to X8 have been cut to correspond exactly in length withrespective audio segments A1 to A8, in order that synchronism can bemaintained between the recorded musical performance and the performanceof the artists in the video segments. Not shown on this display, butavailable elsewhere on the editor, are segments which can be more freelyplaced on the timeline, such as scenery and dramatic elements withoutmusical performance.

The vertical axis as shown is the Y axis, corresponding to focus ondifferent members of the group. Shown in broken outline are clips“behind” the clips arranged in the X and Y directions, indicating the Zdimension, available in more detail by clicking tabs Z2, Z3 etc, as willbe illustrated shortly. The user has chosen to work in only one Z planeat present, and shadow display of these other “layers” can be turned offby a simple control (not shown), if preferred.

The first set of images in the Y direction corresponds in this exampleto shots showing the entire group in performance. Since we are viewingthe Z1 tab, all of these segments are videos of the band performing in astage setting. At other levels in the Y direction, different members ofthe band (Lee, Sam, Nicky) are the focus, for example in a close-up. Notall positions X, Y need to be occupied of course. For example, in theillustration the opening segment X1 of the song has only been shot at agroup level, with no close-ups of individual band members. As in thedramatic examples of FIGS. 4, 5 and 8, rules can be defined for eachsegment, concerning its compatibility with other segments in the X, Y orZ direction. More likely, in the example of a pop promotional video, themain rules at work are those implicit in the assignment of a givensegment to a position on the X, Y and Z axes. For example, a viewer whowants to concentrate on the performance of one particular member of theband can set a constant value Y=3, so that the player will presentclose-ups of band member Sam more often than the other members. Theauthor of the video can satisfy more viewers than conventionally, whereonly one member can be the centre of attention at a given point in thepresentation, while the fan might be more interested in another.

Referring now to FIG. 10, the view has been changed (for example byclicking on the short Z axis illustrated in FIG. 9), effectivelyreorienting the matrix so that the Y axis is compressed into the layers“behind” the main view, vertical axis as displayed on screen thenbecomes the Z axis. The current Y axis layer is then selectable by tabsY1, Y2 etc. across the top of the display. The Tab Y1 is selected,indicating in this example that all of the segments are shots of thegroup as a whole. It will be seen that the matrix is fully occupied,meaning that an entire performance of the group song has been recordedat each of the four locations, stage, studio, castle and beach,represented by values 1, 2, 3, 4 on the Z axis. Again, clicking on anyof these segment icons allows editing of the properties of that and/orediting of the video content itself.

In both views FIG. 9 and 10, the X (time) axis is the horizontal axis,but this is nto necessarily so. It may be very useful in the course ofediting to display a Y-Z matrix, for example. The time axis would thenbe compressed into layers behind the display matrix, the matrixrepresenting a particular time slice. In the illustrated example of themusic video, this view would show what each band member (Y) is doing ateach location (Z) at that point in the song. This gives theeditor/director a good overview o the material available to be used inthe presentation sequence at each point in the song. This view is usefulwhether a polymorphic or standard ‘linear locked’ presentation is beingproduced.

Presentation sequences can be defined by clicking on the icons in thesequence, for example those bounded in bold in FIG. 10. Those sequencescan be stored for use in the final presentation, or one of severalpossible presentations. New sequences can also be generatedautomatically by a command (not illustrated) which will run a simulationof the polymorphic player. These new sequences can be similarlyhighlighted for the director and editor to review. Sequences generatedin this way may be saved by the editor as sequence definitions for laterpresentation, or as a basis for further work. They may be adjustedmanually there and then before being saved, or they may be discarded.

Because the array of options is available to view in different ways, aneasy overview is maintained on the rich set of material available, whichmight otherwise overwhelm the authors and lead to creative opportunitiesbeing missed.

Also by presenting the available content in a pre-defined matrixstructure, a high degree of automation becomes possible. For example, byselecting a clip from among the recorded material and pasting it (suchas by ‘drag and drop’ behaviour), that clip can become automaticallylabelled with its X, Y and Z properties—there is no need for the editoror assistant to type explanatory labels and keep written notes of whereeach clip belongs.

Further illustrations of possible editing facilities will be describedbelow, including different views that may be obtained of the same data,together with functions which facilitate linking of each icon to thecorresponding AN content.

“Content Multiplication”

FIG. 11 illustrates various alternative distribution models, (a), (b)and (c) for polymorphic multimedia content generated according to theprinciples described above. Model (a) in its basic form is the onedescribed above, where the polymorphic production is released on astorage device such as a DVD disc, including on the disc: AN content forall the segments that may be played; a polymorphic rules database bywhich those segments can be assembled and played; and optionally apolymorphic control program, for turning a general-purpose apparatussuch as a PC or programmable DVD into a polymorphic playerimplementation. In return (usually) for payment (indicated “$$$” in thediagram), the user at the right hand side of the diagram receives thiscomplete package of data on a disc for use in their player. Within theplayer, the rules database is accessed under control of a polymorphiccontroller to generate randomised sequences in accordance with the rulesand any parameters set by the user at each occasion of viewing. Apresentation module PRES takes the sequence definition from the sequencegenerator and retrieves and concatenates the AN content accordingly, fordecoding and display to the user as one seamless presentation.

Model (b) is different in that the sequence generation occurs at thesupplier side, the DVD or other storage device carrying only pre-definedsequence definitions SEQ. At the player side, the control is simplified,merely to select between the pre-defined sequence definitions (which maybe only one or several), and the presentation module PRES retrieves theAN content to give the user the desired presentation. While the playerdoes not provided full polymorphic capability, it is still useful in anumber of scenarios. Firstly, several movies have been released indifferent versions, typically a “theatrical release” and a “director'scut”. The example Memento mentioned in the introduction was laterreleased on DVD accompanied by a chronologically ordered version, calledMemento Mori.

Issuing these different versions to consumers on pre-recorded media suchas DVD, or as downloads, conventionally requires duplication of the vastmajority of the A/V content, rather than only the added scenes. Bysupplying the A/V content as a collection of separate scenes, stored indifferent files, together with just encoded sequence definitions foreach version, the novel system allows a multi-version release, whichmight occupy or more discs in today's distribution methods, can bedelivered on one disc only. It is noted that many DVD presentations comewith “deleted scenes”, “alternate endings” and the like, which the usercan access via a menu to view at will. However, none of these presentsthe extra material as part of a continuous, seamless presentation. It isleft to the viewer is left to imagine the context, seriously weakeningthe impact of the presentation. In contrast, using the polymorphicplayer with seamless concatenation in accordance with different sequencedefinitions, numerous versions of the same production can be viewed asentire, seamless presentations, without duplicating the core content.

Additionally, as indicated by the dashed communication lines in FIG. 11(b), a facility may be provided to download further sequence definitions(SEQ'), perhaps in return for a small additional payment, allowing theuser to access different versions without purchasing a whole new DVD. Ina similar vein, it is also possible for some of the sequence definitionspre-stored on the disk to be “locked”, to be unlocked by a digital keyobtained in exchange for some appropriate payment. Payment and downloadmechanisms based on internet connections, or dial-up and satellite/cabledownlinks are well known for pay-per-view purposes, and can be readilyadapted to the new application proposed herein.

It is also possible for the producer/supplier to add additional ANcontent segments to be included in presentations by new sequencedefinitions. These can be supplied on a supplementary disc, withoutre-supplying the basic content. They may also be supplied by downloadthrough the Internet or whatever, as indicated by the broken lines atthe top of the diagram. Local storage such as a hard disk drive isincluded in the user's player, for keeping this additional materialready to be concatenated among segments retrieved from the DVD disc. Aserver at the supplier's side may be arranged automatically to determinewhich additional segments are required to play a new sequencedefinition, or the controller at the user's side may compare thesequence definition with the segments available on the disk and in localstore already, and request the supply of missing segments from theserver at the supplier's side. In this way, artists and commercialpublishers are able to multiply many times the content that theypublish, without multiplying the number of discs. Suppliers and userscan devise many different distribution and payment models, using thefreedom provided by the polymorphic presentation system.

In addition to additional segments recorded at the time of creation ofthe original work, this mechanism allows authors to expand the contentavailable as time goes by. Adding scenes or episodes to stories alreadyrecorded.

FIG. 11( c) illustrates a similar distribution system, working entirelywithout ‘hard copies of the material being bought and sold on disc. AllA/V content, sequence definitions, payments, etc are handled through theInternet or other network (labelled ‘www’ in the diagram). The stepsdescribed with reference to FIG. 11( b) are otherwise the same.

FIG. 11( d) illustrates another implementation, where the presentationsequence is determined and the content assembled seamlessly into aconventional video data stream, all at the server side. The user thenrequires only a standard video player, computer, mobile phone. Theparameters generating the sequence may be selected by the provider (aconventional broadcaster or internet-based service). A broadcaster canvary the presentation, even when ‘repeating’ a movie shown before, forexample, maintaining interest. The same sequence may be shown to allviewers, or different viewers may have different versions of the sameprogram to talk about at work the next morning. In an interactiveservice, either via dial-up and satellite/cable or via internet, theuser may set parameters which are used at the server side to guide theselection of content, just as if the user had the complete system ofFIG. 11( a). The user in that case can use entirely standard and cheaperequipment, while the provide can keep control of the source material,protecting their future revenue.

Combinations of these models (a), (b), (c) and (d) are also possible.For example, the download model (c) may be modified so that the playerdoes include the polymorphic database and sequence generator at theuser's side, rather than accessing only pre-defined sequences. Theentire content is therefore stored in the local storage of the user'sapparatus. The downloads may include the polymorphic control programme,as mentioned in the basic model of FIG. 11( a). It is increasinglycommon for media content to be downloaded to portable video playerdevices, mobile phones and so on, while the cost and availability of abandwidth with such high-volume material remains a limiting factor, to alarge extent. By allowing the apparatus to download only new segmentsand new segments of content required for a new sequence definition, tosupplement the majority of segments already held in local storage, afast and economic supply of very different versions of the same generalcontent can be achieved.

As in the model of FIG. 11( b), it may be determined at the server sideor the user's side, which additional segments are required to supplementthose available already in local memory. Where the sequence definitionis one generated locally by random and/or user parameters, using a rulesdatabase and/or matrix properties of the segments, it may be easier forthe user's apparatus to determine the additional segments required.Identifiers for the missing segments will be contained in the databaseand these identifiers can be sent to the supplier's server in order toretrieve the appropriate segments for download.

AAF-Compliant Implementation

Advanced Authoring Format (AAF) is an industry-driven, open standard formultimedia authoring and post production, created by the AAF Association(see www.aafassociation.org). AAF is a file standard designed to allowthe passage of full information. Not just the video, audio and textmaterial—termed ‘essence’ in the language of AAF—but also the metadatawith the decisions about how material has been manipulated (cuts, DVE,colour correction etc.) and assembled. The metadata also passes onexisting, original information such as timecode or ‘edgecode’,ownership, previous editing. The primary application for AAF is tomaintain this information across as many tasks of post-production aspossible, and to archive it for re-editing material later. Theassumption is still that this rich data format will be ‘flattened’ to aproduction format, fixing the presentation sequence as usual. For ourpurposes, however the AAF format provides an ideal platform for theproduction of rich polymorphic media. Since AAF is an open standard, andis ready-made to accept data extensions and ‘plug-ins’ extensible, italso provides an ideal platform for the development of authoring toolsand playback tools.

FIG. 12 illustrates the basic elements of an AAF file, which mightdefine one or more scenes of a production, or a complete production. Thefile can contain any amount of video and audio source material, labelledSRC1(VID) and SRC1(AUD) respectively. The file also contains one or more‘Material Objects’ or ‘Mobs’, of which MOB1 and MOB2 are examples. EachMob contains a reference to parts of the source materials defining asegment of a/v presentation (video and audio ‘essence’). As shown by thedotted link, the Mob may also include a reference to source materialsSRC2(VID) and SRC2(AUD) which are not stored within the file itself. TheMob also contains the metadata recording what processing has beenapplied, edit history and so forth. The AAF file structure is such thatparts of it, particularly the metadata, can be edited and re-writtenwithout re-writing the whole file. The metadata will typically includeEDL data, of the type shown in FIG. 2, all in a standard format. Themetadata can also be extended however to contain data such as the rulesapplicable to the segment in a polymorphic movie system. Provided aplayback apparatus can read these rules and apply the appropriateprocessing as described already above, the AAF file can be used as themedium for delivery of the content and the rules database forpolymorphic movies.

FIG. 13 shows in its top part the functional structure of anAAF-compatible authoring module for polymorphic movies, in accordancewith a preferred embodiment of the invention. The module accesses theAAF database, including source (essence) materials and metadata, andallows polymorphic authoring operations based on several different viewsof the material and its properties. In a first view, the user can viewand edit the rules explicitly which govern assembly of the presentationsequence. In other views, more global changes can be defined, in thecategory of shot type (examples might be ‘extreme close-up’, ‘close-up’,‘wide shot’, ‘dolly shot’), story structure. A special view is providedto the design and selection of endings, and another for referencingfootage of the same event from different camera angles. Depending on thedegree of polymorphism and design freedom allowed in a givenimplementation, these views may allow changes within rigid parametersonly, or may allow changes affecting the whole presentation, includingwhat is seen in other views. For example, changes made in the ‘ShotType’ view may translate implicitly into rules that are generatedautomatically, but can be seen and edited directly in the ‘Rules’ view.The views shown are not the only ones possible, but merely a selectionof ones which the author might want intuitively to use.

FIG. 14 shows just one of these views, the Rules view, in a suitableuser interface display. At the foot of the control pane, a presentationsequence of shots is represented S1, S2 up to S6. These shots are beingassembled into one scene using a two-dimensional matrix. Referring alsoto FIG. 3 now, it can be appreciated that polymorphism is possible bythe selection of shots within each scene, as well as at the level ofscene selection within the movie as a whole. It will also be seenshortly how polymorphism can readily be introduced at the frame level,that is within shots themselves. The polymorphic authoring system caninclude general editing facilities to define these shots and scenes outof the source material. Alternatively, and particularly since the AAFformat is recognised by some of the popular digital editing toolsalready on the market, these shots will typically be imported as AAFMobs already formed in some other application. In either case, thesource materials can be brought in from their source files convenientlyusing a ‘drag and drop’ interface, using a pointing device and thumbnailimages on the display screen. The job using the present authoring systemthen becomes one of linking the shots and scenes together with thedesired polymorphic constraints and possibilities.

If the polymorphic rules and matrix structure have already been defined,at least partially, during a scripting phase, the matrix may alreadyexist. That is likely to be the case when a project is conceived fromthe outset as a polymorphic production. In that case, the filming andrecording of content will be done to generate shots which already havetheir place in the matrix and can be imported into the editingapplication relatively automatically. On the other hand, when producinga polymorphic movie from pre-existing material, the task of collectingthe material and defining the rules to go with it in the editingapplication are more likely to be done simultaneously.

In the state of the display as shown, the user has selected shot S2 fordetailed attention, causing the editor interface to display y-axisalternatives for the shot. From simple S2 to (S2, Y4). The user has thenopted to display and define the specific rules stored for shot (S2,Y2).A predefined menu of parameters is displayed, including:

-   -   ANY NEXT?: A ‘yes’ here would indicate no restriction on the        shots that can follow immediately after this one. The user        selected ‘no’.    -   AND ONLY: The user can specify the that a certain shot must        follow the present one.    -   NOT: The user can specify certain shots with which the present        shot cannot must not appear in the same presentation.    -   VARIABLE CUT: The user can specify a range of frames within        which a ‘variable cut’ can be made (see explanation below).    -   SLIDING CUT: The user can specify a range of frames within which        a ‘sliding cut’ can be made (see explanation below).    -   TRANSITION: The user can specify a form of transition to the        next shot. Examples are ‘cut to’, ‘fade to black’, ‘dissolve’.        This recognises that the type of transition is part of the        character of the presentation, and might be different for        different versions of the same shot.

As in the example of FIG. 3, these are only a few typical examples ofthe kinds of rules that may be applied. Rules may be forward- and/orbackward-looking.

Sliding Cut

FIG. 15 illustrates the operation of a ‘sliding cut’, which is a simpletechnique to provide frame-level polymorphism, which can be controlledto give subtle or dramatic changes to the viewing experience. Theexample illustrates the transition from shot S1 to shot S2. A range offrames SC at the end of shot S1 is designated as a sliding cut range.The transition to shot S2 can be made at any point in this range,depending on a random value and/or a user-set parameter. In onepresentation, the shot S1 might end very early, as shown at C1. Onanother viewing, the shot might linger to point C2.

Note that the sliding cut alters the overall length of the presentation.A global variable, either random or user defined, could be assigned tobring forward all the sliding cuts so as to present a hurriedpresentation, or to delay the sliding cuts to the maximum for a morelingering effect. The author can define the sliding cut range so as notto change any significant action or expression, leaving the plot intact,but the rhythm changed, The author could equally define the sliding cutrange so that sometimes a significant event or facial expression will beseen on some viewing occasions and not seen on others.

A sliding cut can in principle be defined to vary the entry point to asegment, as well as the exit point illustrated in FIG. 15. addinganother option to the menu. The available transition points may bedefined as a free selection within a range of frame numbers, as shown inthe illustrations, or may be defined as a set of specific framepositions or sub-ranges (for example “SLIDING CUT: 100-124, 142,150-172”). The principle of the sliding cut can be applied to vary thepoint of transition between whole scenes, as well to vary the point oftransition between shots within one scene. Although referred to as asliding ‘cut’, this is merely shorthand: the type of transition might bea dissolve, fade etc.

Variable Cut

FIG. 16 illustrates a feature similar to the sliding cut, which altersthe timing of transition from shot S1 to S2, without altering theoverall length of the scene. The transition may occur in the video only,with dialogue on the audio channel continuing underneath. Again, theauthor defines a range of frames VC within which the cut can bedetermined by random values or user parameters. The cut may occur early,as shown at C1, at an intermediate point C2 or be delayed as at C3.

As with the sliding cut, the available transition points may be definedas a free selection within a range of frame numbers, as shown in theillustrations, or may be defined as a set of specific frame positions.Again, the exact form of transition may be a cut, dissolve etc.

These frame level variations allow quite subtle changes in the viewingexperience. For more radical changes, one could also include thefacility to program ‘sliding’ or variable changes at a scene or shotlevel. To visualise this, imagine that each small unit shown in thetimelines in FIGS. 15 and 16 might be a scene or a shot, rather than aframe. This might be too radical, if significant action is present inthese scenes. On the other hand, movies often include several‘establishing shots’ to let the viewer know where the action is takingplace, and subtle variations in the viewing experience might be obtainedby adding or losing one or more of these.

Content Capture

In conventional production, much work is done to prepare a shot list andto keep track of which shots, required in the final presentation, havebeen filmed and which haven't, how many takes of each one have beenfilmed, and which take is to be used. A lot of manual work is requiredbefore editing can begin, bringing shots from different sources,identifying them and their place in the production, on a conventionaldisplay or on a novel “matrix” display as illustrated in FIGS. 9, 10 and14.

FIG. 17 illustrates exploitation of the “matrix” representation,described already in the editing system, extended into thepre-production phase, and used to automate the capture of a material ina most efficient manner. The display is similar to that seen in FIG. 10.The display is created prior to filming, however. Therefore at thisstage, rather than thumbnails, the icons in the matrix are merely placeholders for content when available. Each item in the display mayrepresent an AAF file or a MOB within an AAF file. Before the materialhas been filmed, the MOB may contain only meta-data, detailing itssubject, perhaps linking to a “storyboard” image, script file,continuity notes or the like, but importantly holding a place in thegraphic representation of the production displayed on the screen shownin FIG. 17.

The inventors propose that this interface, in addition to being a placeof work for the editor in post-production, should become the templateinto which content is imported, preferably during filming itself, but atleast during linking and/or transfer of material from tape to editingsuite. Accordingly, the system is provided with a data connection to thecamera/tape recorder which captures the digital video segments. The filmcrew use this display interactively to select the item which is going tobe filmed, as illustrated by the pop-up menu and cursor shown in FIG.17. When “add take” is selected, meta data is created already to receivethe material about to be filmed. (As explained above, the essence itselfmay be kept in a separate file, with the AAF file itself only receivinga link to the location of that take. Alternatively, the data may beimported completely into the AAF file, according to the wishes of thedesigner.)

It is not uncommon for six takes of the same shot to be taken routinelyin a production, giving a so-called “shooting ratio” of 6:1. Once theseshots have been taken, the user can click the second option on the menu“view/select take”, reviewing and selecting which of the takes is to beused for that shot at the end of the presentation. The take selectioncan be done during filming, or of course deferred until later. Even inthe former case, exploiting the rich metadata capabilities of the AAFfile format, the alternate takes can remain linked, to be revisitedlater if desired.

Three distinct shadings are shown on the icons in the display of FIG.17, indicating that colour coding, or other highlight devices can assistthe crew in keeping track of which segments have been filmed and whichhave not. In a simple scheme illustrated in FIG. 17, dark hatching isused to indicate shots which have been imported, and the take selected.The audio tracks, in the case of a pop promotional video, are thestarting points, so they are naturally filled with dark hatching also.Lighter hatching indicates scenes where one or more takes have beenimported, but it remains to select which take is to be used in thepost-production phase. Blank squares indicate segments defined duringthe scripting/storyboard phase, for which there may be metadata, but noA/V content has been imported yet. Provided that the system is beingused at the time of filming, rather than merely to import filmedmaterial from tapes, this visual cue may save expensive mistakes, whereit is discovered that one or more scenes have been missed.

Further forms of colour/highlight may indicate, for example, scenes forwhich raw content has been attached, but editing of that segment hasstill to be done. This distinction may alternatively be saved for adifferent view, to simplify the “import content” display.

Whether or not the matrix display is used actually at the time offilming, it can still be used to import film content directly into thepre-established matrix structure, or other rules structure. Theprinciples of organising the importation of content into pre-establishedlocations within a AAF data structure and display can be applied morewidely than the specific polymorphic production environment describedabove. That is to say, even for a “conventional” multimediapresentation, there is still the opportunity to organise the datastructure at the scripting/storyboard stage, into which filmed segmentscan be linked automatically at the time of filming, rather than havingto be picked manually from a jumble of material at a later stage. Aswith FIG. 10, alternate views of the production can be obtained byclicking on different tabs, Y1, Y2 etc, and by changing from Z to Y axisrepresentation. At the import stage, an additional dimension is the‘take’ number which may be may not be flattened before entering the nextstage, but can be opened again so long as the links to the alternatetakes are maintained in the metadata.

Further Comments

While the embodiments above are presented in the form of a database ofrules and a Sequence Generator processing random numbers with referenceto those rules, the invention also encompasses the possibility ofintroducing automation in our segments to the extent that they areconstructed to behave as automatons in their own right. The sequencegeneration task then becomes distributed to the individual segments,each segment being processed according to its rules to bond itself withother segments and so define the presentation sequence. Segments mayeven be enabled to regenerate and evolve or even modify themselves. Justlike computer viruses do have certain proliferating abilities, there isno reason to believe that segments could not perform self-perpetuatingtasks, within the confines of the materials and rules assigned to them.Imagine a Segment as a complete scene initially with cut-in and cut-outprocedures built in, but eventually able to evolve in quasi-Al(artificial intelligence) entities able to draw from libraries carriedon the DVD, on a remote server or in the players themselves, whichseamlessly play an array of instructions and functions like transitioneffects, dissolves, cuts, variable/sliding transitions, randomisationand so on. Because these instructions are digital, we are in a positionto randomise and eventually even allow each cell or segment to recogniseif it fits in a particular sequence or if by reference it needs supportor exclusion of itself or other cells in said sequence, in order tomaintain the integrity of the story, whatever it turns out to be.Therefore the prerequisites of a PM are segments that are digital informat, can be randomised and can carry seamless in/out transitions

The invention in other aspects does not necessarily include thegeneration of new sequences at the time of playback, at the user's side,or at the server side. Although these areas are the main application ofthe editing and content capture systems described, it will be see thatthose sophisticated tools will be useful in the production ofconventional, linear multimedia forms, as well as polymorphic. Ifpossible, a disc carrying the polymorphic movie should carry a ‘locked’version as well, for compatibility with non-PM players. The editor canjust as well produce a separate version for that market, however, instandard DVD or HD-DVD/BluRay™ format.

The skilled reader will appreciate that the examples presented hereinare only a small selection, representative of a very wide range ofpossibilities for the construction of authoring systems and players forpolymorphic media. The invention encompasses the above examples,together with many other variations and modifications that may beenvisaged by the person skilled in the art from reading of thisdocument, or developed in the course of experimentation with real mediacontent.

1-61. (canceled)
 62. An apparatus for polymorphic presentation ofpre-recorded audiovisual content, the recorded content comprising aplurality of individual segments, the apparatus comprising a sequencegenerator operating on a media player, for defining a presentationsequence of a subset of the segments and for causing the segments to bepresented in accordance with the defined presentation sequence, wherein:said sequence generator uses randomization to define at least part ofthe presentation sequence, each segment involved in the randomized partof the sequence has associated with it at least one rule ofcompatibility with other segments, and the sequence generator isoperable to generate said randomized part of the sequence by selectingand adding segments to a sequence already partly defined, using (i)randomized values and (ii) predetermined rules specifying compatibilitybetween the segments already included in the sequence and segments whichare candidates for adding to the sequence.
 63. An apparatus as claimedin claim 62 wherein the sequence generator is operable to select only asubset of the recorded segments to include in the presentation sequence.64. An apparatus as claimed in claim 62 wherein the sequence generatoris operable to vary the relative order of segments in the presentationsequence in accordance with a randomized value.
 65. An apparatus asclaimed in claim 62 wherein the sequence generator is responsive to anOUT rule for a segment, the OUT rule restricting which other segmentscan immediately follow the segment.
 66. An apparatus as claimed in claim62 wherein the sequence generator is responsive to an IN rule for asegment, the IN rule restricting which other segments the segment canimmediately precede.
 67. An apparatus as claimed in claim 62 wherein thesequence generator is responsive to rules defining a group of segmentsnominally occupying the same position on a first axis, the sequencegenerator selecting the group according to a first randemised randomizedvalue, and then selecting a particular member of the group according toa second value.
 68. An apparatus as claimed in claim 67 operable suchthat the second value can be a randomized value or a user-determinedparameter.
 69. An apparatus as claimed in claim 62 wherein the sequencegenerator is responsive to a rule for a segment restricting which othersegments can appear at any point after the segment.
 70. An apparatus asclaimed in claim 62 wherein the sequence generator is responsive to arule for a segment restricting which other segments can appear in thesame presentation sequence.
 71. An apparatus as claimed in claim 62wherein the sequence generator is operable to build presentationsequences in an order selected based on information stored with thepre-recorded content.
 72. An apparatus as claimed in claim 62 whereinthe sequence generator is operable to apply fuzzy rules, the outcome ofsuch rules depending on a comparison between the relative strengths ofcontradictory rules, or on a comparison of a rule strength and arandomized value.
 73. An apparatus as claimed in claim 62 wherein thesequence generator is responsive to further attributes stored inrelation to certain segments.
 74. An apparatus as claimed in claim 73wherein a segment may have an attribute of dominance, the dominancepermitting a rule to be broken according to a strength of the rule. 75.An apparatus as claimed in claim 62 wherein the sequence generator isoperable to re-use a previously-generated randomized value in subsequentdecisions between candidate segments.
 76. An apparatus as claimed inclaim 62 wherein the sequence generator is operable such that one ormore of the included segments is itself defined by a presentation ofsub-segments assembled at the time of playback.
 77. An apparatus asclaimed in claim 62 wherein the sequence generator may be responsive toa rule associated with one of a segment and a sub-segment permittingtransition to or from the segment at a number of alternative points,selecting the point of transition in the presentation sequence inaccordance with a randomized value.
 78. An apparatus as claimed in claim77 wherein the sequence generator is responsive to a rule defining asliding cut between two segments, such that a segment is terminatedearly or late depending on a randomized value, the sliding cut thushaving an effect on the total length of the presentation sequence. 79.An apparatus as claimed in claim 77 wherein the sequence generator isresponsive to a rule defining a variable cut between two segments, inwhich the point of transition is varied in accordance with therandomized value, without altering the overall duration.
 80. Anapparatus as claimed in claim 62 wherein the sequence generator isoperable to retrieve said rules from special metadata fields within anAdvanced Authoring Format (AAF) file which also defines thecorresponding audiovisual content.
 81. (canceled)
 82. A computerreadable medium as claimed in claim 86 wherein the audiovisual contentand the rules database are stored in one or more AAF files, the rulesdatabase being stored as metadata within the AAF format.
 83. (canceled)84. A method for production of polymorphic audiovisual presentationscomprising: assembling on a storage medium (i) a number of pre-recordedsegments of audiovisual source material and (ii) a database of rules foruse by a sequence generator operating on a media player for defining apresentation sequence of a subset of the segments and for causing thesegments to be presented in accordance with the defined presentationsequence, wherein: said sequence generator uses randomization to defineat least part of the presentation sequence, each segment involved in therandomized part of the sequence has associated with it at least one ruleof compatibility with other segments, and the sequence generator isoperable to generate said randomized part of the sequence by selectingand adding segments to a sequence already partly defined, using (i)randomized values and (ii) predetermined rules specifying compatibilitybetween the segments already included in the sequence and segments whichare candidates for adding to the sequence.
 85. A method for polymorphicpresentation of pre-recorded audiovisual content, the recorded contentcomprising a plurality of individual segments, the method comprising:defining, by a sequence generator operating on media player, apresentation sequence of a subset of the segments; and causing thesegments to be presented in accordance with the defined presentationsequence, wherein defining the presentation sequence comprises usingrandomization to define at least part of the presentation sequence,wherein each segment involved in the randomized part of the sequence hasassociated with it at least one rule of compatibility with othersegments, and said randomized part of the sequence is defined byselecting and adding segments to a sequence already partly defined,using (i) randomized values and (ii) predetermined rules specifyingcompatibility between the segments already included in the sequence andsegments which are candidates for adding to the sequence.
 86. A computerreadable medium storing computer program instructions capable of beingexecuted by a computer processor on a computing device, the computerprogram instructions for polymorphic presentation of pre-recordedaudiovisual content, the recorded content comprising a plurality ofindividual segments, the computer program instructions defining thesteps of: defining, by a sequence generator operating on a media player,a presentation sequence of a subset of the segments; and causing thesegments to be presented in accordance with the defined presentationsequence, wherein defining the presentation sequence comprises usingrandomization to define at least part of the presentation sequence,wherein each segment involved in the randomized part of the sequence hasassociated with it at least one rule of compatibility with othersegments, and said randomized part of the sequence is defined byselecting and adding segments to a sequence already partly defined,using (i) randomized values and (ii) predetermined rules specifyingcompatibility between the segments already included in the sequence andsegments which are candidates for adding to the sequence.
 87. Acomputer-readable medium storing (a) audiovisual content and (b) a rulesdatabase comprising rules, wherein the audiovisual content comprises aplurality of individual segments, and wherein the rules of the rulesdatabase enable a sequence generator operating on a media player todefine a presentation sequence of a subset of the segments, whereindefining the presentation sequence comprises using randomization todefine at least part of the presentation sequence, wherein each segmentinvolved in the randomized part of the sequence has associated with itin the rules database at least one rule of compatibility with othersegments, and said randomized part of the sequence is defined byselecting and adding segments to a sequence already partly defined,using (i) randomized values and (ii) predetermined rules in the rulesdatabase specifying compatibility between the segments already includedin the sequence and segments which are candidates for adding to thesequence.
 88. A computer-readable medium as claimed in claim 87, whereinthe audiovisual content and the rules database are stored in one or moreAAF files on the computer-readable medium, the rules database beingstored as metadata within the AAF files.
 89. A computer-assistedauthoring tool arranged to generate a computer-readable medium thatstores (a) audiovisual content and (b) a rules database comprisingrules, wherein the audiovisual content comprises a plurality ofindividual segments, and wherein the rules of the rules database enablea sequence generator operating on a media player to define apresentation sequence of a subset of the segments, wherein defining thepresentation sequence comprises using randomization to define at leastpart of the presentation sequence, wherein each segment involved in therandomized part of the sequence has associated with it in the rulesdatabase at least one rule of compatibility with other segments, andsaid randomized part of the sequence is defined by selecting and addingsegments to a sequence already partly defined, using (i) randomizedvalues and (ii) predetermined rules in the rules database specifyingcompatibility between the segments already included in the sequence andsegments which are candidates for adding to the sequence.